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Introduction 


The  project  aims  to  develop  a  battery  of  tests  to  diagnose  visual  dysfunc¬ 
tions  in  cognitively  impaired  observers  based  on  their  oculomotor  reflexes. 
The  oculomotor  reflexes  provide  a  simple  and  robust  method  to  study  vision 
in  passive/immature/impaired  observers.  For  example,  oculomotor  reflexes 
are  widely  used  to  study  vision  in  infants  and  in  marketing  research.  The  pro¬ 
posed  method  is  versatile,  applies  to  a  wide  range  of  visual  mechanisms,  and 
can  provide  both  qualitative  (yes/no)  and  quantitative  (degree)  estimates  of 
the  visual  loss.  The  project  embodies  a  full  research  and  development  cy¬ 
cle,  from  establishing  the  most  effective  stimuli  for  various  kinds  of  visual 
dysfunctions  to  the  design  of  a  computerized  testing  kit  suitable  for  use  by 
non-specialists.  After  a  short  training  course,  the  kit  could  be  used  at  army 
bases  and  local  hospitals  to  make  a  quick  initial  diagnosis. 


2 


Body 


Task 

▼  1)  Hardware  development 

•1.1)  Shopping  for  hardware 

•  1 .2)  Testing  the  hardware  kit 

•  1 .3)  Finalizing  the  hardware  kit 

▼  2)  Software  development 

•  2.1)  Stimulus  presentation 

software 

•  2.2)  Data  acquisition  software 

•  2.3)  Data  analysis  software 

•  2.4)  Adaptive  stimulus 

presentation  sofware 

•  2.5)  Data  analysis  and 

diagnosis  inference 

▼  3)  Visual  tests  development 

•  3.1)  Identify  target  visual 

dysfunctions 

•  3.2)  Design  the  corresponding 

visual  stimuli 

•  3.3)  Pilot  testing  of  the  stimuli 

•  3.4)  Test  stimuli  for  normal 

subjects 

•  3.5)  Test  stimuli  for  subjects 

with  visual  and/or 
cognitive  dysfunctions 
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Figure  1:  Gantt  chart  for  the  project. 

As  specified  in  the  Statement  of  Work,  the  main  goal  of  the  first  12 
months  of  the  project  was  to  design  effective  stimuli  allowing  the  oculomotor 
reflexes  to  be  used  as  diagnostic  tools  for  various  visual  dysfunctions.  The 
three  reflexes  proposed  for  this  project  (optokinetic  reflex,  pupillary  reflex, 
orienting  reflex)  form  a  complementary  set.  For  example,  our  preliminary 
data  demonstrated  that  optokinetic  reflex  in  response  to  drifting  gratings 
is  an  effective  tool  for  testing  various  low-level  visual  mechanisms,  such  as 
visual  acuity,  contrast  sensitivity,  color,  and  depth  sensitivities.  On  the  other 
hand,  orienting  and  pupillary  responses  to  salient  stimuli  can  be  used  to 
test  higher-order  visual  functions,  such  as  shape  perception,  face  recognition, 
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visual  memory,  etc.  Even  if  the  subject’s  eye  movements  are  impaired  by  the 
injury,  the  pupillary  reflex  alone  can  be  used  as  a  qualitative  diagnostic  tool 
for  both  the  low-level  and  the  high-level  visual  functions. 

Figure  1  shows  a  Gantt  chart  indicating  project  tasks  and  their  current 
state  of  completion.  Overall,  all  the  current  goals  were  met:  we  selected 
and  purchased  all  the  necessary  hardware  for  the  project,  identified  common 
visual  dysfunctions,  designed  pilot  stimuli  testing  for  these  dysfunctions,  and 
developed  the  necessary  software  to  present  the  stimuli,  capture  the  eye¬ 
tracking  data,  and  analyse  it  for  the  relevant  eye-events.  These  tasks  are 
described  in  detail  in  the  following  sections. 


Hardware  development 


Figure  2:  Two  ISCAN  Inc.  eye-trackers  used  for  the  project.  The  head-mounted 
unit  is  mounted  on  a  safety  goggles  frame.  The  remote  unit  is  positioned  next  to 
the  monitor. 

We  primarily  investigated  different  models  of  eye-trackers  in  the  first 
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quarter.  The  goal  was  to  find  the  model  best  suited  for  the  project,  i.e., 
portable,  and  well  suited  for  experimental  work  with  cognitively  impaired 
subjects.  Other  search  parameters  included  programming  interface  capabil¬ 
ities,  ease  and  speed  with  which  hardware  customizations  could  be  made, 
and  the  overall  cost.  After  investigating  several  eye-tracker  models  being 
used  on  Northeastern  University  campus  and  visiting  local  manufacturers 
of  eye-tracking  equipment  we  purchased  two  video-based  eye  trackers  from 
ISCAN  Inc.  (based  locally  in  Woburn,  MA):  one  in  remote  and  one  in  head- 
mounted  configuration  (Figure  2).  This  allowed  us  to  test  both  eye-tracking 
modes  and  to  find  the  one  best  suitable  for  this  project. 

The  head-mounted  version  can  be  mounted  on  a  goggles  frame  or  a  base¬ 
ball  cap.  It  appears  to  be  the  preferred  solution  and  will  be  used  for  most 
subjects.  Head-mounting  the  eye-tracker  allows  to  study  subjects  with  un¬ 
controllable  head  and  body  movements.  The  remote  (desktop  mounted)  ver¬ 
sion  with  automatic  head  tracking  will  be  used  for  those  subjects  which 
might  have  difficulty  with  or  might  not  tolerate  wearing  the  head-mounted 
version.  This  version  will  also  be  used  for  measuring  binocular  functions  for 
all  subjects,  because  it  allows  to  track  both  eyes  simultaneously. 

The  eye-tracking  units  came  with  a  laptop  and  desktop  computers.  The 
laptop  computer  is  more  portable,  but  currently  can  only  be  used  for  120 
Hz  eye-tracker  sampling  rate.  The  desktop  can  be  used  for  240  Hz  sampling 
rate,  but  is  less  portable.  Pilot  data  indicates  that  120  Hz  is  adequate  for 
our  purposes,  but  we  are  leaving  the  240  Hz  option  open,  because  the  higher 
sampling  speed  might  be  necessary  for  noisier  data  (subjects  with  droopy 
eyelids,  thick  glasses,  uncontrolled  head- movements,  etc.). 

To  ensure  maximum  performance  two  computers  (laptops)  are  used  in  our 
current  hardware  configuration.  One  computer  is  dedicated  to  controlling 
the  eye-tracker  and  collecting  the  data.  Stimulus  generation,  presentation, 
and  data  analysis  are  carried  out  by  another  computer.  The  two  computers 
communicate  via  serial  connection. 


Visual  tests  development 

In  the  second,  third,  and  forth  quarters  we  collected  and  analyzed  pilot  data 
for  normal  observers.  Pilot  stimuli  included  various  drifting  gratings  and 
odd-ball  images  of  faces,  bodies,  and  words.  Figure  3  shows  some  examples. 
The  images  were  obtained  from  freely  available  research  databases.  Typical 
pilot  session  lasted  6  minutes  during  which  30  -  40  visual  stimuli  were  shown. 
Subjects  were  viewing  stimuli  on  a  21”  LCD  monitor  from  a  distance  of  60  - 
80  cm.  The  head-mounted  version  of  the  eye-tracker  was  used  for  all  subjects, 
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Figure  3:  Examples  of  visual  stimuli  used  for  the  pilot  study.  Drifting  gratings 
(top  left)  were  used  to  evoke  optokinetic  reflex,  neutral  and  mildly  offensive  words 
(top  right)  were  used  to  evoke  pupillary  response,  images  of  odd-ball  and  mildly 
arousing  female  figures  were  used  to  evoke  orienting  reflex  (saccades)  and  pupillary 
response. 


eye-movements  were  sampled  at  120  Hz.  At  this  stage  the  data  were  collected 
off-line:  recorded  on  a  hard  drive  during  the  experimental  run  and  analyzed 
after  its  completion.  The  pilot  data  was  used  to  optimize  the  visual  stimuli 
and  to  design/test  eye-event  filters  as  described  below.  A  fragment  of  the 
pilot  data  for  one  subject  is  shown  in  Figure  4. 

The  stimuli  were  used  to  evoke  reflex  eye-movements  in  26  normal  ob¬ 
servers.  We  received  clearances  from  the  Northeastern  University  IRB  and 
MEDCOM  USAMRMC  for  work  with  human  subjects  in  March  2011.  PI 
and  his  graduate  students  were  used  as  subjects  prior  to  this  date.  We  used 
inexperienced  observers  naive  to  the  purpose  of  the  test:  observers  only  knew 
that  they  were  taking  part  in  an  eye-tracking  experiment.  The  only  instruc¬ 
tions  received  from  the  experimenter  were  to  attend  to  the  screen.  Such 
experimental  conditions  were  chosen  to  simulate  testing  cognitively  impaired 


6 


observers.  Such  observers  will  be  tested  in  the  final  phase  of  the  project. 


Software  development 

Once  the  eye-tracking  units  arrived  in  January  2011,  we  started  software 
development  for  visual  stimulus  presentation,  eye-tracking  data  readout,  and 
the  data  analysis. 

Visual  stimulus  presentation  and  data  import 

A  software  library  for  adaptive  visual  stimulus  generation,  presentation,  and 
analysis  (C++  and  OpenGL  based)  was  developed  in  the  course  of  the  first 
and  second  quarters.  This  library  is  currently  used  to  create  visual  stimuli 
and  to  collect  real-time  data  from  the  eye-tracker.  The  data  is  received  from 
the  eye-tracker  computer  via  serial  connection.  The  library  incorporates  a 
graphical  user  interface,  which  allows  to  easily  observe  eye  events  and  to  vary 
the  stimulus  parameters. 

Eye-event  filters 

Using  the  collected  pilot  data  we  designed  software  filters  for  automatic  de¬ 
tection  of  the  relevant  eye  events  in  real  time.  Altogether,  4  types  of  filters 
were  developed:  blinks,  saccades,  optokinetic  reflex  (nystagmus),  and  pupil¬ 
lary  response  (contraction/dilation).  The  filters  are  based  on  measuring  and 
thresholding  eye-movement  velocities  and  pupil  area  changes.  The  filters 
were  designed  to  be  used  with  real-time  data:  they  are  fast  and  they  do  not 
require  large  amounts  of  data  to  work.  About  one  second  of  data  is  suffi¬ 
cient  to  detect  saccades  and  pupillary  reflex,  and  several  seconds  of  data  -  to 
detect  optokinetic  reflex  (which,  typically,  lasts  for  3  -  5  seconds  at  a  time). 

The  filters  were  validated  using  human-parsed  fragments  of  the  pilot  data. 
The  collected  eye-tracking  data  were  searched  and  parsed  into  different  types 
of  relevant  eye  events,  such  as  blinks,  visual  nystagmus,  saccades,  and  pupil 
dilations.  Half  of  the  data  were  used  to  design  the  software  filters,  the  other 
half  were  used  for  filter  testing.  Filter  performance  for  the  high- amplitude  eye 
events  relevant  to  our  project  was  close  to  100%  dropping  below  90%  for  very 
noisy  data  only.  Identifying  saccades  was  the  task  most  affected  by  noise. 
Noisy  data  can  result  from  poorly  adjusted  eye-tracker  parameters.  Such 
adjustments  are  particularly  important  when  subjects  are  wearing  glasses, 
have  droopy  eyelids,  or  narrow  eyes.  Altogether,  the  filters  performance  was 
found  adequate  for  the  purposes  of  our  project. 
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Figure  4:  A  fragment  of  raw  eye-tracking  data.  Eye-event  labels  generated  by 
software  filters  are  overlaid  on  the  eye  position  and  pupil  size  traces. 


Figure  4  illustrates  the  filters  performance.  It  shows  45  seconds  of  raw 
eye-tracking  data  for  one  subject.  The  top  (black)  trace  plots  the  horizontal 
eye  position,  the  middle  (blue)  trace  plots  the  vertical  eye  position,  the  bot¬ 
tom  (green)  trace  plots  the  pupil  size.  Saccades  appear  as  abrupt  changes  in 
the  eye  position  (vertical  or  horizontal),  nystagmus  -  as  the  sawtooth  profile 
for  the  horizontal  position,  and  pupillary  reflex  as  a  fast  change  in  the  pupil 
size.  These  events  were  well  captured  by  the  corresponding  filters,  which  is 
demonstrated  by  the  overlaid  event  symbols.  For  example,  the  last  quarter 
of  the  recording  (prior  to  the  blink)  shows  a  nystagmus  event  (indicated  by 
the  dashed  cyan  line)  followed  by  a  medium-sized  horizontal  saccade  accom¬ 
panied  by  a  fast  pupil  dilation  -  contraction  event  (magenta  and  cyan  solid 
lines).  These  events  were  evoked  by  a  full-screen  drifting  color  grating  fol¬ 
lowed  by  one  second  empty  screen  and  then  a  semi-nude  image  of  Marilyn 
Monroe  displayed  in  the  right  part  of  the  screen. 


Currently,  the  filter  development  is  in  its  pilot  stage,  i.e.,  the  filters  were 
implemented  in  MATLAB  and  applied  to  off-line  data  only.  The  next  stage 
will  be  to  implement  the  filters  as  a  part  of  the  stimulus  generation  and 
analysis  C++  software  library  and  to  test  them  on  the  data  collected  in  real 
time.  This  stage  will  be  completed  by  the  end  of  2011.  Once  this  part  of  is  in 
place,  we  shall  start  working  on  making  the  stimuli  adaptive,  the  adaptation 
procedure  being  governed  by  the  automatic  eye-event  registration. 
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Key  Research  Accomplishments 


•  After  studying  the  market  of  eye-tracking  solutions  we  identified  and 
purchased  two  models,  which  can  be  used  together  to  carry  out  robust 
eye-tracking  for  patients  with  cognitive  disabilities. 

•  A  software  library  for  adaptive  visual  stimulus  generation,  presentation, 
and  data  acquisition  was  developed. 

•  Pilot  visual  stimuli  were  designed  and  tested  on  26  normal  observers 
naive  to  the  purpose  of  the  test.  No  specific  instructions  were  given  to 
simulate  experiments  with  cognitively  impaired  observers. 

•  The  collected  data  was  used  to  optimize  the  visual  stimulation  and  to 
design  software  filters  which  will  allow  automatic  detection  of  relevant 
reflex  eye  movements  in  real  time. 
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Reportable  Outcomes 


There  were  no  reportable  outcomes  yet. 
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Conclusions 


We  successfully  developed  and  implemented  a  pilot  battery  of  tests  designed 
to  diagnose  various  visual  dysfunctions.  The  principal  hardware  and  software 
components  of  data  acquisition  and  analysis  pipeline  are  in  place  now.  The 
visual  stimulation  and  data  collection  units  were  tested  on  26  naive  observers 
with  normal  vision.  We  used  the  collected  data  to  design  software  filters  for 
automatic  detection  of  oculomotor  reflexes.  These  results  mostly  complete 
the  first  phase  of  the  project  and  allow  us  to  start  the  next  phase,  which 
focuses  on  two  goals:  making  the  developed  battery  of  visual  tests  adaptive, 
and  testing  a  large  number  of  normal  subjects  to  establish  the  distribution 
of  responses  for  people  with  normal  (reference)  visual  functions. 
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